Infinity Engine Talk Table Editor
---------------------------------
Copyright(C) 2005-2006 Baronius

E-mail:  baronius@blackwyrmlair.net
Website: http://www.blackwyrmlair.net/Tools/tlkeditor.php
Forum:   http://forums.blackwyrmlair.net/index.php?showtopic=1336

Readme for Version: 1.02


TLKEditor CREDITS
-----------------

Baronius (author, source code, design)
dragon_lord (tester)
Vlad (bug report)

TLKEditor is based on the 1.12 version of TLKViewer. TLKViewer CREDITS:
	Baronius (author, source code, design)
	dragon_lord (comprehensive testing, excellent suggestions)
	jastey (tester of the search engine, useful suggestions)
	Rabain (tester, useful suggestions on the program's design)
	SirKill (tester of the early version)
	Vlasak (tester, great ideas such as 'remember last tlk', suggestions on the graphs and statistics)
	igi (user suggestions)

The description of the StrRef type flag was taken from the IESDP (http://iesdp.gibberlings3.net)

TLKViewer's Search Engine is based on the "Working with extremely large text files" article of Douglas Tietjen:
htp://www.delphipages.com/news/detaildocs.cfm?ID=41
(Also available at http://www.yourdelphi.com/article_detail.php?articleid=13)


TABLE OF CONTENTS

I.   Features
II.  The editor
III. Description of functions
	1. Open a .tlk file
	2. Search
	3. Statistics
	4. Export
	5. The TLKAssistant
IV.  Knowns issues
V.   Planned features
VI.  Version history





I. Features

- Works with all of the IE games (BG1, PST, IWD, BG2, IWD2), including all of their official expansions and unofficial mods
- It can an edit talk table (TLK) files (edit existing strings and sound references and any number of new ones)
- It treats each string individually (you can restore any string you changed since your last save)
- Fast saving & loading
- It can truncate the TLK file
- Customisable browser window to display the string (font, font color & size, background color), displays the offset, length, type, sound reference, volume variance and pitch variance of the string
- Fast and powerful search engine with several settings (complex search with multiple search terms, highlight results, case sensitivity, whole words, length-oriented and absolute search method, search from a certain string, theoretically unlimited number of results)
- Technical statistics of your TLK file (average string length, longest string, number of strings by type, number of strings by their size, size map and string size graphs)
- Text statistics of your TLK file (occurances of user-defined words/expressions)
- TLK to TXT export (instant raw export, guided export with various settings)
- Configuration (your last used settings are stored and remembered when the program is run next time, auto-load last opened file when the program starts, button to load last opened file)




II. The editor

You can browse the TLK file by using the Arrow buttons, but you can also type the StrRef index of the string you wish to display. When you have typed the number, click 'Refresh' or the text window, for instance.
VIEW IN NEW WINDOW -- this button will open a new window. It's also a browser, with a greater text box and several other settings such as the background, color and font of the text.
The 'Find Next' button does the same as the one in the program's main window (check section III below to see how search works), but obviously the matching string will be displaying in this window (as well).
The Color and Font Settings can be used to customize the text window. If you would like to specify the background color and the text color as well, click the last radiobutton called 'Custom', and then choose the colors you are happy with.

You can edit the selected string in the text window. CTRL+Z, the default undo hotkey of Windows certainly works, it will undo the last operation you did with the text. TLKEditor provides additional editor options for you:
- Alt+R -- If you have edited the currently selected string, you can completely undo your changes on it by pressing Alt+R. It will revert the string to its last saved state. You can use this option at any time on any string you have changed earlier (provided you didn't SAVE the changes into the TLK file meanwhile.). If the selected string is a new string made by you (i.e. it did not exist when you saved the file last time), TLKEditor will offer to delete it.
- Alt+Z -- Complete restoration. This will do the same as Alt+R but will all strings, automatically. It will also ask whether you want to delete your newly made strings.

You can create a new string by clicking the 'New' button or by pressing Alt+N. TLKEditor will automatically set the cursor's position to the editor window, so you can start typing immediately after the new string has been created.

You can save your changes into the TLK file by pressing F5 or Alt+S. This will apply all of your changes on the existing strings and will save your newly created strings as well. Your earlier changes will be written to the TLK and obviously you will not be able to undo these earlier changes once you have saved the TLK file.
TLKEditor supports backups and 'Save As'. These options as well as all of aforementioned functions belong to the TLKAssistant. You can find detailed information about at III./5. of the readme.

NOTE: you can encounter underlined text at certain parts of TLKEditor (in the main window, the 'Search Methods' and the 'Type' of the selected string are marked in this way). If you click these, TLKEditor's help window will appear, with an explanation on the given function/term.




III. Description of functions

The main menu is on the top left corner of the program's main window. Move the mouse cursor over one of the four major menupoints to display the settings of the given function. They will appear in the middle of the Settings box.
On the bottom left of the window, you can see the buttons and settings of the Standard Search option.


1. Open a TLK file

A.) Open last opened TLK -- loads the most recent .tlk file you opened with TLKEditor. This function will obviously not work when you are running the program for the first time, or if the file has been moved since the program's last run.
Notes: 
- The TLK file cannot be opened and loaded if it is Read-Only. If you get a file open error and the TLK file's Read-Only attribute is true, set it to False and try to load the file again.
- TLKEditor does not lock the TLK file after it has been loaded, so you can open it in another program as well at the same time. However, since the content of the TLK file is cached by TLKEditor when the file is loaded, no further changes on the original TLK file made by using an external editor or another running instance of TLKEditor will appear in TLKEditor unless you re-load the TLK file. You can re-load the TLK file by clicking the "Open last opened TLK file" button.


B.) Auto-open last opened file when TLKEditor starts -- if this option is enabled, the most recent .tlk file you opened will always be loaded automaticaly when TLKEditor is run.


2. Search

Both Standard and Advanced Search use the following options:
- Highlight matches -- if this option is enabled, the matching text (the search result) will be highlighted in the string that contains the search result.
- Case sensitivity -- if this option is enabled, the difference between capital case and lowercase will not be ignored
- Whole words -- if this option is enabled, the search terms will be considered whole words, and the search process will be performed according to this
- Search Whole talk table -- if this option is enabled, the search engine will search for the given text in the whole strings setction of the .tlk file
- Search from String -- if this option is enabled, and a StrRef index is given, the search engine will search for the given text only from the string with the specified StrRef index.
- Search Method -- when searching for the same expression multiple times (in case of the Standard Search, constantly clicking the Find next button, ) the Absolute method will include the characters of the previous search result during the next search, while the Length-oriented won't.  For example, if you are searching for 'haha', and the search engine encounters 'hahahaha', the Absolute method will consider this as three matches, while the Length-oriented will find two matches only. The type of the Search Method has higher signifance when using the Advanced Search or when calculating Statistics.

A.) Standard Search
Type the search term (which is a string of any size theoretically) to the 'Text to Find' field. If you are happy with the settings (highlight, case-sensitivity, whole words, search method, search from given string), click the Find Next button. The result will be displayed in the window of the selected string on the right (i.e. the matching string will automatically be selected). Next clicks on the 'Find Next' button will display the next occurances of the text if there is any. 
Below the 'Find Next' button, you can see the current search position with bold text (absolute position in the strings section of the talk talk table). If you have specified a string for the 'Search from String' option, the current search position will be the offset of the specified string automatically.
If you change any of the search settings, the search will be restarted when the 'Find Next' button is clicked.

B.) Advanced Search -- move the mouse cursor over the 'Search...' button, and when the settings appear in the middle of the Settings box, click the 'Advanced Search' radiobutton. Now click the 'Search...' or the newly displayed 'Advanced Search...' button, and a new window will appear.
Let's start with the bottom of this window. The settings on the bottom right are the same as in case of the Standard Search (see above). The most important part is on the bottom left: the terms of the complex search. Complex search will find all strings that match the search criteria. The criteria can be any number of strings of (theoretically) any size.
Let's add some search terms! Click the field above the 'Add this to the list' button, and type a word (or any text, text part, anything). Click  the aforementioned 'Add this to the list' button. The text will appear in the window above the field you were typing to. You can add any number of texts in this way. If case-sensitivity is enabled, the program will accept the same text more times if there is a difference between them in capital- and/or lowercase.
Now you can select the type of filter (you can also change this after the search has been completed). In the small rectangle on the bottom, there are two radiobuttons. The first is checked by default. If 'Display strings that contain ALL of the expressions on the list' is checked, the program will display those strings which contain EVERY given search term. For example, if you add 'ogre', 'die,' and 'TROLL' as search terms, the program will find a string such as 'Die ogres and trolls!' but won't consider 'An ogre and a troll!' or 'Die ogre!' as matching strings. The other option ('Display strings that contain ANY of the expressions on the list') will display those strings that contain ANY of the texts: in case of the above example, any strings that contain at least one of the words 'Ogre', 'troll' or 'die' would be displayed as matching strings.
Click the 'START SEARCH' button.  Unless you have specified very common words as search terms, the search should take only for a few seconds. The results will be displayed on the top left of the window, in a listbox. Select any item of the list to display the corresponding string on the window on the top right. The browser in TLKEditor's main window is also updated, and the 'View in new window' function also works. If the highlight option of the Advanced Search (which is independent of the similar option of the Standard search) is enabled, you can highlight the matching texts in the string by clicking the 'Highlight (next) match' button. The matching texts will be highlighted one after another. You can re-set the highlight position to the beginning of the string by clicking the 'Reset' button.

The filter can be changed at this time either ('Display strings that contain ALL of the expressions' / 'Display strings that contain ANY of the expressions'), and the listbox on the top left will be updated -- no need to repeat the search! To start an entirely new search, click the 'NEW SEARCH' button. The search results will be deleted, and the button's caption will change back to 'START SEARCH'.


3. Statistics

You must save your changes before using this function.
Two checkboxes, two options. Technical statistics will provide information on the size of the talk table, the total number of strings, the average string length and so on. Word/expression statistics is similar to Advanced Search, but it does not display the matching strings (since it is not a search function). It will count the occurances of the specified text. Both technical and text statistics can work with user-defined values.
When you click the 'Statistics' button, it will calculate with default settings. The results will be displayed in several windows.

A.) Technical statistics
The first page displays general data (filename, number of strings between given sizes etc.). You can specify custom criteria. To see the new results, make sure to get the statistics re-calculated.
The second page is for Size Graphs. The Size Map of talk table displays how the size of strings changes from the first to the last string. The StrRef indexes is one the x axis, while the size is on the y axis. You can 'zoom' the map (view a smaller segment of StrRefs) by specifying the boundaries in the custom fields. You can see a bigger (and thus a more accurate) version of the graph by clicking the 'More accurate graph' button. If the difference between the two limits is smaller than 25, the pillars of the diagram will get labels, with exact values on them (the y value, i.e. the exact size of the given string). The second graph displays the number of strings by their size. The 'zooming' function as well as the 'More Accurate Graph' button also works here. If you change any of the limits in the custom fields, click the 'REFRESH GRAPH' button to display the new diagram.
The third and fourth page shows the number of strings based on their type. You can see the IESDP (or click the underlined 'Type' property of the selected string in TLKEditor's main window) for more information on the valid StrRef types that are used by the Infinity Engine games. Here you can also specify custom statistic criteria, but it does not make much sense.

B.) Word/Expression statistics
Text statistics is the last page (or the only page if Technical statistics is not enabled). There are twelve expressions here, and TLKEditor counts their occurances in the .tlk file. Obviously you can modify these fields. If you do so, click  the 'Re-calculate only this page' or the 'Re-calculate statistics' button (the latter one will re-calculate all statistics including the technical stats as well if they are enabled). It takes awhile to count the occurances of all of the 12 expression. If you are curious about one word/expression only, fill the twelfth field and click the 'Re-count only the 12th' button.
In case of word/expression statistics, Whole words and Case-sensitivity are turned on by default. You can change these settings (I don't think it makes too much sense), click the 'Unlock settings' button to enable the change of default settings.

NOTE:  Specifying very common words (such as 'you', 'your') in the custom fields will result in slightly longer calculation times (1-2 minutes). One-character strings are not recommended (and it doesn't make much sense to make such calculations...), they should work, but it isn't guaranteed that you won't get a stack overflow error.


4. Export

You must save your changes before using this function.
This function will export the strings section of the .tlk file into a text (.txt) file. 

A.) Raw Export -- when this option is checked, the destination file will contain the strings of the .tlk without ANY changes: one string after another. Raw Export creates the text file instantly. 

B.) Guided Export -- when this option is checked, you can specify the structure of the target .txt file.
- Index Strings -- strings will be numbered in the text file, starting from zero.
- Write empty strings too -- strings that contain no text will also be written to the text file.
- Spacelines -- will add empty lines between strings (results in a clearly arranged text file).


5. The TLKAssistant

TLKAssistant contains all of the functions that support editing. You can activate it via the button on the bottom right part of TLKEditor's main window or by pressing F2.

You can see the state of the currently selected string at the top of its window. It can be 'Original', 'Edited' or 'Created by User'. If it is an edited or newly created string, the 'Revert this string to original (Alt+R)' button is enabled. It will revert the string to its last saved state. You can use this option at any time on any string you have changed earlier (provided you didn't SAVE the changes into the TLK file meanwhile.). If the selected string is a new string made by you (i.e. it did not exist when you saved the file last time), TLKEditor will offer to delete it. 
The 'APPLY ALL changes and SAVE talk table file (ALT+S or F5)' will save your changes into the TLK file. This will apply all of your changes on the existing strings and will save your newly created strings as well. Your earlier changes will be written to the TLK and obviously you will not be able to undo these earlier changes once you have saved the TLK file.

The 'UNDO ALL changes made since the last save' button will restore the talk table file (TLKEditor's string & strref cache) completely. This will do the same as the Revert String function but will all strings, automatically. It will also ask whether you want to delete your newly made strings.

The 'TRUNCATE talk table' button is capable of erasing all strings from the talk table from a specific string, to the end of the file. For example if you truncate the TLK file from string 10001 and it has 75000 strings, the new file will have 10000 strings (65000 strings will be erased). You can use this option only if all of the strings are in Original state (i.e. the TLK file has been saved and no new changes have been made).
This can be a dangerous option if you don't know what you're doing. You will probably never need it anyway.

You can see four checkboxes on the bottom of TLKAssistant's window:

'Save to different file' -- when you're saving the talk table, TLKEditor will open a Save As window for you instead of writing the data directly to the TLK file you have been editing.

'Make a backup before updating an already existing TLK file' -- TLKEditor will make a <filename>.bak file (most likely dialog.tlk.bak or dialogf.tlk.bak) before overwriting (updating) an existing TLK file.

'TLKAssistant always stays on top' -- the window of TLKAssistant will always remain 'on the top of the screen', it will have the highest priority over all of the visible windows. If you don't use high resolution, this option is not recommended. You can always activate it by pressing F2 anyway.
NOTE: You can move TLKAssistant to any position on the screen. TLKEditor will remember its position when you run it next time.

'Display TLKAssistant when a TLK file has been loaded' -- TLKAssistant will automatically appear when you open a TLK file. 

When a TLK file is open in TLKEditor, you can always activate TLKAssistant by pressing F2 or the 'TLKAssistant (F2)' button on the bottom of TLKEditor's main window.




IV. Known issues
NONE




V. Planned features (inherited from TLKViewer v1.12)
- Browse and edit dialog.tlk and dialogf.tlk at the same time
- Search with other criteria (StrRef type, associated sound etc.)
- Text export from a given string (and not the entire talk table)




VI. Version history

15.9.2006  -- version 1.03
- Fixed: error when using advanced search after saving the talk table

11.1.2006  -- version 1.02
- Fixed: tlkeditor "duplicated" newlines for BG2
- Fixed: problems when switching from ASCII mode to text mode in browser

20.11.2005  -- version 1.00
- Initial release.